Improving storage performance with bcache in a virtualization scenario

The development of hard disk technology has not kept up with the general speed-up of computers and modern day technological developments like virtualization and cloud deployment. Flash storage, on the contrary, improves access speed and bandwidth significantly but at comparably higher costs. As in most scenarios only a fraction of the actual data stored on disk is ever used in arbitrary operation, a combination of both storage technologies can offer a good balance of speed and capacity. This resulted in tiered storage solutions offered by a range of commercial system providers. If the procurement of a new storage system is out of question, a Linux based flash cache approach can significantly improve performance of existing systems with moderate efforts in terms of setup and configuration. This paper describes how an overloaded storage system primarily used as NFS based storage system for virtualization hosts could be brought back to operation by a multilayer data caching strategy to overcome high latencies and a low throughput. The resulting system design combines the benefits of high capacity slow spinning disk storage with flash only storage to a hybrid system. The presented approach was tested and evaluated in a real world productive environment. Various performance measurements and data are presented.